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EXECUTIVE SUMMARY 



■ • The Carl D. Perkins Vocational and Applied Technology Act of 1990 requires states 
to design and implement accountability systems tint include a core of performance measures 

■ and standards for vocational education program? at the secondary and postsecondary levels. 

■ The Perkins Act also calls for the provision of quality vocational-technical education 
m programs for all students, especially members of special populations. While states are 

■ accountable for developing and monitoring the performance measures and standards, local 
recipients of Perkins dollars must implement the accountability system. This document is 

I designed to help local and state educators improve their accountability systems and develop 

program improvement plans where called for. 

This handbook is organized into three parts. Part I focuses on (1) measuring the 

■ success of programs serving students who are members of special populations and (2) 
* improving programs that do not meet the established standards. It offers guidelines on how 
^ to enact a single, comprehensive accountability system that satisfies the requirements in 
M Sections 115, 1 17, and 1 18 of the Perkins Act. Part I also contains a discussion of incentives 

or adjustments within accountability systems that are designed to encourage service to 
M targeted groups or students with special needs. In addition, Part I provides examples of how a 

secondary and postsecondary accountability system are implemented. It presents activities 

■ prescribed under the Perkins Act for improving the education of all students including those 
from special populations. 

™ Part II provides examples of performance measures and standards that were adapted 

m from selected states' accountability systems. Other measures and standards can be developed 

■ based on these examples. Part III presents definitions of relevant terms taken from the 
Perkins Act Final Regulations and suggested operational definitions gleaned from some 

■ statewide accountability systems. Selected sections from the Glendale Union High School 
District and Minnesota's State Board of Technical Colleges Accountability Systems are 

I included as appendices. 

■ The implementation of an accountability system is intended to be an ongoing process 
™ of system development and improvement to better serve the nation* s diverse student 
m populations. It is vital that vocational educators meet the challenge of developing 
| comprehensive accountability systems, use them to identify program strengths and areas 

where improvement is needed, and then implement good improvement plans. 

_ i 
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INTRODUCTION 

Congress clearly expected the Carl D. Perkins Vocational and Applied 
Technology Education Act of 1990 (hereafter referred to as the Perkins 
Act) to significantly improve the vocational education opportunities of 
students who are members of special populations. To improve programs 
and outcomes for all students, especially those who are members of special 
populations, a statewide accountability system of standards md measures 
was to have been implemented by the 1992-1993 school year. While state- 
level personnel (guided by the Committee of Practitioners) are responsible 
for developing and monitoring the standards and measures, local recipients 
of Perkins dollars must implement the accountability system. This document 
is designed to help local and state educators improve their systems of 
performance standards and measures and develop program improvement 
plans where called for. 

By Fall 1993, all states have complied with the mandate to develop 
standards and measures. Some states have established very comprehensive 
standards and measures while others have adopted a small number. 
Additionally, various states require local recipients to collect varying 
amounts of data about students and programs. It should be remembered 
that local recipients and states need to collect sufficient data to satisfy the 
Section 117 requirement that the findings of the annual evaluation be 
reviewed with the full and informed participation cf representatives of 
individuals who are members of special populations in order to 

identify and adopt strategies to overcome any barriers which are 
resulting in lower rates of access to vocational education programs 
or success in such programs for individuals who are members of 
special populations; and 

evaluate the progress of individuals who are members of special 
populations in vocational education programs assisted under this 
Act. (Section 117(1)) 

It is only logical for states to enact a single, comprehensive 
accountability system. Obviously, such a system would be easier to 
administer and would provide local educators with clear-cut objectives. 
Such a system should 

• comply with the system of core standards and measures of performance 
(Section 115); 



The Perkins Act 
mandates the 
implementation of a 
statewide accountability 
system of measures and 
standards. 



Barriers to access and 
success must be 
identified and 
overcome. 



States must consider the 
requirements of Sections 
115, 117, and 118 in 
designing their 
accountability systems. 
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• provide local recipients with the framework of an evaluation system 
(Section 117); 

• gather evidence that the local and state recipients are meeting the 
assurances of service and equal access for members of special 
populations (Section 118); and 

• satisfy any additional local and state data needs. 

Congress also recognized that outcomes for schools and/or programs 
with larger concentrations of special populations will probably be less 
positive than those for programs with more advantaged students. To assist 
such programs, the Perkins Act includes the requirement that the 
accountability system include "incentives or adjustments that are designed 
to encourage service to targeted groups or special populations" (Section 
1 15(b)(3)). It is imperative that this mandate be strictly followed lest the 
legislation result in grave disservice to the very students it is most meant to 
serve. 

Use the Section 118 States have implemented their new accountability systems and the first 

assurances when year of evaluations was completed in 1993. Programs which fall short of 

developing a program meeting, or of making "substantial progress" toward meeting, the established 
improvement plan. . . . , 

standards are expected to develop a program impiovement plan, including 

strategies to overcome barriers to access and success in quality vocational 

education programs (Section 1 17( 1)). The assurances enumerated in Section 

1 1 8 of the Perkins Act will be a useful reference for local recipients who 

need to develop a program improvement plan because they have failed to 

meet the state standards. The assurances in Section 1 1 8 are meant to ensure 

that members of special populations have access to quality vocational 

programs and to the services essential for success. It is probably a safe 

assumption that local recipients who have failed to meet accountability 

standards have also failed to provide access and services required by Section 

118. In order to assist these local and state educators in improving their 

accountability systems and developing program improvement plans, this 

publication has been organized into the following three parts: 

Parti 

• Guidelines to ensure comprehensive accountability systems 

• An explanation of how to implement the systejn 

• A discussion on using the assurances found in Section 1 18 to develop 
a program improvement plan 
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• A discussion, with examples, of incentives and adjustments 
Part II 

• Examples of general standards and measures 

• Examples of standards and measures for special populations 

Part III 

• A listing of definitions concerning special populations provisions 
garnered from the Perkins Act Final Regulations, and suggested 
operational definitions gleaned from state publications 

SECTION 115: STATE AND LOCAL STANDARDS AND MEASURES 

The Perkins Act mandates that each state develop and implement a 
system of performance standards and measures for vocational education 
programs. Section 1 15 of the Perkins Act specifies the inclusion of student 
measures of both academic and occupational outcomes in statewide systems. 
In addition, the accountability system must include incentives or adjustments 
designed to encourage service to targeted groups or special populations. 

Section 1 15, State and Local Standards and Measures, states 
(a) GENERAL AUTHORITY. Each state board receiving funds under 
this Act shall develop and implement a statewide system of core 
standards and measures of performance for secondary and 
postsecondary vocational education programs. Each State board 
receiving funds under this Act, before the expiration of the thirty-day 
period beginning on the date of the enactment of the Carl D. Perkins 
Vocational and Applied Technology Education Act Amendments of 
1990, shall appoint the State Committee of Practitioners (in this section 
referred to as the 'Committee') as prescribed by section 512(a) after 
consulting with local school officials representing eligible recipients, 
and representatives of organized labor, business, superintendents, 
community-based organizations, private industry councils established 
under section 1 02(a) of the Job Training Partnership Act, State councils, 
parents, special populations, correctional institutions, the administrator 
appointed under section 1 1 1 (b)( 1 ), the State administrator of programs 
assisted under part B of the Education of the Handicapped Act, the 
State administrator of programs assisted under chapter ) of title I of the 
Elementary and Secondary Education Act, the State administrator of 
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"Each system . . . shall 
include . . . measures of 
learning and 
competency gains, 
including student 
progress in achievement 
in basic and more 
advanced academic 
skills...." 
PERKINS ACT 
Section 115 



programs for students of limited English proficiency, and guidance 
counselors. Such systems shall be developed and implemented before 
the end of the two-year period beginning on the date of the enactment 
of the Carl D. Perkins Vocational and Applied Technology Education 
Act Amendments of 1990 and shall apply to all programs assisted under 
this Act. Eligible recipients may make local modifications to such 
system based on economic, geographic, or demographic factors, or the 
characteristics of the population to be served. Such modifications shall 
conform to the assessment criteria contained in the State plan. The 
State board shall convene the Committee on a regular basis to review, 
comment on, and propose revisions to a draft State proposal, which the 
State board shall develop, for a system of core standards and measures 
of performance for vocational programs. 

(b) REQUIREMENTS. Each system developed under subsection (a) shall 
include 

(1) measures of learning and competency gains, including student 
progress in the achievement of basic and more advanced academic 
skills 

(2) one or more measures of performance, which shall include only 

(A) competency attainment 

(B) job or work skill attainment or enhancement including student 
progress in achieving occupational skills necessary to obtain 
employment in the field for which the student has been 
prepared, including occupational skills in the industry the 
student is preparing to enter 

(C) retention in school or completion of secondary school or its 
equivalent and 

(D) placement into additional training or education, military 
service, or employment. 

(3) incentives or adjustments that are 

(A) designed to encourage service to targeted groups or special 
populations, and 

(B) for each student, consistent with the student's individualized 
education program developed under section 614(a)(5) of the 
Education of the Handicapped Act, where appropriate and 

(4) procedures for using existing resources and methods developed in 
other programs receiving Federal assistance. 
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SPECIAL POPULATIONS: GUIDELINES TO ENSURE 
COMPREHENSIVE ACCOUNTABILITY SYSTEMS 

The following are offered as guidelines to ensure that provisions for 
special populations become an integral part of the accountability systems 
mandated by the 1990 Perkins Amendments: 

1. Accountability systems must contain good outcome and process data 
concerning special populations. Obviously, a program's outcomes 
provide evidence of its effectiveness. For example, completion and 
retention rates of a dropout prevention program will reveal much more 
about the success of the program than counting the number of books in 
the library or computer/student ratios. Yet, process data (e.g., enrollment 
data, number of students receiving specific services, computer access) 
should help explain outcomes, both good and bad. 

2. Incentives and/or adjustments to programs designed to encourage the 
participation of students who are members of special populations must 
be integrated into all vocational programs. (Incentives and adjustments 
are discussed on pp. 11-14.) 

3. Measures of gain should be used to encourage local educators to recruit 
a d serve special populations. Hoachlander, Levesque, and Rahn( 1992) 
suggest that, including measures of gain in addition to measures of 
attainment is a strategy that will discourage the exclusion of members 
of special groups (or creaming). Gains or "value added" measures 
provide change over time versus attainment measures that establish an 
achievement level. Hoachlander and his associates point out that while 
attainment measures are very necessary, gains measures allow flexibility 
between advantaged programs and less advantaged programs. For 
example, the academic attainment level of an inner city school or 
community college with a student population consisting largely of 
special populations may be much lower than that of a student population 
of a suburban school with more advantaged students. The achievement 
levels of students in different situations may vary greatly and put some 
teachers and students in an unfavorable light. Student gains measured 
by pretests and posttests may be a better determinant of program success. 

4. The accountability system should produce data on all special groups 
identified in the 1990 Perkins Amendments. These groups include the 
following: 



Include good outcome 
and process data. 



Integrate incentives and 
programmatic 
adjustments into all 
vocational programs. 

Include measures of 
gains, in addition to 
measures of attainment. 



Identify and gather data 
on all groups. 
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Disaggregate data. 



Apply standards to all 
students. 



Use Section 118 
assurances when 
developing program 
improvement plan. 

Assess students. 



Conduct ongoing 
evaluation. 



Students who have limited-English proficiency (LEP) 
Students who are educationally and/or economically disadvantaged 
Students with disabilities 

Students who are in programs designed to eliminate sex bias 
Students who are incarcerated 

Examples of these kinds of data would be data on the numbers of LEP 
students placed in jobs in their field and the number of students with 
disabilities who have successfully completed program "X," and so on. 

5. Data should be gathered by gender for each subgroup as well as the 
total population (e.g., numbers of females and males with disabilities 
who complete vocational program "X")- 1 

6. Data should be gathered to address situations unique to the state, locality, 
or even institution. For example, some areas may have a high 
concentration of a special population(s) (e.g., a large number of 
Vietnamese students who have limited-English proficiency). 1 

7. Standards should be constant for all students in a program, including 
those who are members of special populations. The intent of the 
legislation is to provide supplemental services to raise the achievement 
level of students who are members of special populations, not lower 
program standards. Similarly, validity and reliability measures in 
assessing program and student outcomes should apply to all groups, 
including special populations. 

8. When a review of the evaluation findings necessitates the development 
of a program improvement plan, Section 118 assurances (see pp. 15- 
19) should be used as guidelines. 

9. The accountability system should include assessment data from all 
students, including those who are members of special populations. 
(Reminder: Some members of special populations need special testing 
accommodations.) 

10. There should be an ongoing review of the accountability system to 
ensure that the standards and measures are nonbiased and fair to all 
students. 



1 Section 117 requires a review of the findings by a group which must include full 
participation of a representation of individuals who belong to special populations. The data 
must provide information on those groups in order to determine access and success. 
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SECTION 117: PROGRAM EVALUATION AND IMPROVEMENT 

Based on the statewide system of performance standards and measures, 
each recipient of Perkins funds must evaluate the effectiveness of their 
programs. If as a result of the evaluation a program is not making 
"substantial progress" toward meeting the established standards and 
measures, a program improvement plan must be developed. Section 117 
also mandates that local recipients review the findings of the evaluation in 
order to identify and adopt strategies to overcome any barriers to access 
and success with the full and informed participation of representatives of 
individuals who are members of special populations (e.g., personnel who 
work with LEP students, disabilities, and Chapter 1 programs). 

Section 1 17, Program and Evaluation and Improvement, states 

(a) ANNUAL EVALUATION. Each recipient of financial assistance under 
part C of title II shall annually evaluate the effectiveness of the program 
conducted with assistance under this Act based on the standards and 
measures (or modifications thereto) developed as required by section 
115. As part of each such evaluation, each such recipient shall 

(1) review programs, with the full and informed participation of 
representatives of individuals who are members of special 
populations in order to 

(A) identify and adopt strategies to overcome any barriers which 
are resulting in lower rates of access to vocational education 
programs or success in such programs for individuals who are 
members of special populations; and 

(B) evaluate the progress of individuals who are members of special 
populations in vocational education programs assisted under 
this Act; and 

(2) evaluate the progress of vocational education programs assisted 
under this Act in providing vocational education students with 
strong experience in and understanding of aM aspects of the industry 
the students are preparing to enter. 

(b) LOCAL PROGRAM IMPROVEMENT PLAN. Beginning not less 
than 1 year after the implementation of the provisions of section 1 1 5, if 
any recipient described in subsection (a) determines that the recipient 
is not making substantial progress in meeting the standards and measures 
developed as required by section 115, such recipient shall develop a 
plan, in consultation vv^th students, parents, and teachers concerned, 
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for program improvement for the succeeding school year. Such plan 
shall describe how the recipient will identify and modify programs 
funded under part C of title II, including 

(1) a description of vocational education and career development 
strategies designed to achieve progress in improving the 
effectiveness of the program conducted with assistance under this 
Act; and 

(2) if necessary, a description of strategies designed to improve 
supplementary services provided to individuals who are members 
of special populations. 

(c) STATE AND LOCAL JOINT PLAN. If, after 1 year of implementation 
of the plan described in subsection (b), sufficient progress in meeting 
the standards and measures developed as required by section 115 has 
not been made, the State shall work jointly with the recipient and 
students, parents, and teachers concerned to develop a plan for program 
improvement. Each such plan shall contain 

( 1 ) a description of the technical assistance and program activities the 
State will provide to enhance the performance of the eligible 
recipient; 

(2) a reasonable timetable to improve the school performance under 
the plan; 

(3) a description of vocational education strategies designed to improve 
the performance of the program as measured by the evaluation; 
and 

(4) if necessary, a description of strategies designed to improve 
supplementary services provided to individuals who are members 
of special populations. 

(d) FURTHER ACTION. The State shall, in conjunction with the eligible 
recipient, annually review and revise the joint plan developed under 
subsection (c) in order to improve performance and will continue to do 
so each consecutive year until the recipient sustains, for more than 1 
year, fulfillment of the State and local standards and measures developed 
under section 115. 
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FIRST STEPS 

As local and state administrators attempt to implement the mandated 
accountability system, gaining faculty and staff support is vital. Taking 
proactive steps can avert negative reactions to change efforts at the grassroots 
level. The following actions are recommended to facilitate the 
implementation of the accountability system of performance standards and 
measures specified in the Perkins Act: 

1. When implementing the accountability system of standards and 
measures, faculty and staff should take part in the planning and decision- 
making process where their input is sought and valued. A good 
professional development program fosters collaboration and encourages 
the faculty and staff to become stakeholders in the new initiative. 

2. To prepare the faculty and staff for the task ahead, extensive inservice 
activities are essential. With adequate instruction, they are likely to be 
more confident when confronted with problems as they perform their 
responsibilities. In addition, appropriate ongoing professional activities 
serve to keep the participants updated and on task. 

3. The institution should examine local, state, and federal accountability 
requirements and practices to determine how to align the accountability 
system of performance standards and measures with the existing 
accountability program. If possible, the accountability system of 
standards and measures should be established as a part of the overall 
program of evaluation and accountability. 

4. During the planning stage of the system of performance standards and 
measures, it is important to determine what data is to be gathered and 
how the needed data is to be collected. 

5. Equally significant to systematic program evaluation is deciding who 
will review the data and what evaluation procedures will be utilized. 
The availability of relevant data can lead to effective program 
improvement. 



Involve faculty and staff 
in the planning and 
decision-making 
process. 



Provide extensive 
inservice activities. 



Align the core of 
performance measures 
and standards with the 
existing accountability 
program. 



Determine how and 
what data to gather. 



Decide who will review 
the data and what 
evaluation procedures 
will be utilized. 
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Examples 



Secondary Level: Glendale Union High School District 

Vocational teachers, coordinators, and administrators from the Glendale 
Union High School District in Phoenix, Arizona, have established an 
accountability system based on their state's performance standards and 
measures. This team of professionals, after evaluating their first steps, 
decides and plans together for the next steps. Suggestions are made on 
how to better serve students and make learning in the classroom worthwhile. 
More importantly, focusing on and enhancing the positive makes everyone 
who is involved in the entire process feel like winners. As a result of the 
evaluation, programs that do not meet the core standards and measures 
have to develop their own program improvement plans. The Glendale Union 
High School District sees this evaluation as the beginning of an ongoing 
process — all in the best interest of students. 

In 1992-1993, the Glendale Union High School District, under the 
direction of Dean Petersen, Vocational Education and Special Programs 
Coordinator, took the following first steps to implement its vocational 
accountability system. The emphasis throughout was on program 
improvement in conjunction with state and federal mandates. 



implementing the vocational accountability system based on the state's 
performance standards and measures. The orientation provided teachers 
with background information on standards and measures including 
legislative explanations and priorities set in the state of Arizona. 

The district's overall educational evaluation plan was also presented 
during this meeting, with an explanation concerning how the two 
systems would fit together. Ample time was allowed for participants 
to voice concerns and clarify any misunderstanding or confusion. Roles 
of teachers and administrators were discussed and clarified. It was 
evident by the end of the session that everyone had a stake in 
implementing a good, viable, accountability system. 

2. Next, the school district formed a steering committee composed of eight 
vocational teachers and coordinated by Dean Petersen. Committee 



Win teachers ' and 
administrators ' support 
in the implementation of 
the accountability 
system. 



Prior to the opening of classes for the 1992-1993 academic year, the 
district provided a four-hour general orientation session for the district's 
vocational education personnel. The program was designed to enlist 
the support of the district's vocational teachers and administrators in 
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members were volunteers who felt personally invested in the task ahead. 
The committee, which met regularly throughout the school year, 
gathered information from faculty and administrators in order to make 
plans and decisions on the vocational evaluation. Updates on each of 
the meetings were disseminated to other vocational teachers and 
administrators for more feedback. This group of professionals worked 
diligently to pave the way for the smooth implementation of the local 
evaluation plan. 

3. The Glendale Union High School District conducted its training and 
staff development activities in February and March of 1993 to prepare 
teachers and administrators for the work ahead. 

First, vocational education teachers, administrators, and coordinators 
of special populations programs attended a two-hour training session 
on ihe entire accountability process. During this session, local 
evaluation teams with a designated team leader were formed to guide 
the process in each school. The evaluation teams consisted of at least 
one representative from the following groups: students with disabilities, 
students who are disadvantaged, and LEP students. Also included were 
people representing programs to promote gender equity, business and 
industry representatives, and teachers. Every local team member made 
a two-year commitment to ensure stability in team composition and 
eliminate further training of new members. Teams had a minimum of 
eight members to a maximum of eleven. A forum was held at a later 
date to discuss various dimensions of the implementation process and 
to familiarize each member with the locally developed resource guide 
designed to assist local evaluation teams. 

4. Finally, the local evaluation teams oversaw the evaluation of the 
vocational programs in their respective schools. By this time, all the 
district's vocational personnel were aware of the process, the 
information to be gathered, and how the information would be reviewed 
and used to improve their programs. 

The intensive process included the following stages: 

(a) one-day on-site evaluation which included in-depth interviews with 
students and teachers 

(b) one-day of reviewing and corroborating all data 

(c) one-half day of drawing local improvement plans 
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Conduct follow-up 



activities. 



Conduct statewide 
assessment. 



To provide each school the benefit of an objective evaluation, vocational 
program area teachers from each school participated as a member of a 
local evaluation team for another school. 

5 . Asa result of the evaluation, schools in the Glendale Union High School 
District are currently intensifying follow-up activities for their graduates 
to assure good placement. They consider this phase to be critical in the 
successful implementation of their evaluation system. 

The vocational personnel in Glendale Union High School District have 
shown that they can make a difference. By working together with a common 
purpose, they are able to improve their programs and services to their 
students. (See Appendix A for selected sections of the Glendale Union 
High School District: Vocational/Technical Education Performance 
Standards: Local Evaluation Team Resource Guide.) 

Postsecondary Level: Technical Colleges in Minnesota 

In 1993, Minnesota's State Board of Technical Colleges (SBTC) 
implemented an accountability system that incorporates all the elements 
required by the Perkins Act. The SBTC, with the Committee of Practitioners' 
review and approval, designed their accountability program to provide 
quality vocational-technical education programs through continuous 
program improvement for the diverse student populations served by each 
of their technical colleges. 

The system's core of performance measures is based on a statewide 
assessment conducted in 1992 by SBTC to determine if the technical colleges 
are successfully recruiting, retaining, graduating, and placing their students. 
Instructors and administrators from the state's technical colleges were 
involved in the assessment through discussion, meetings, interviews, and 
surveys. The assessment effort resulted in the following six measures: 2 

1 . Number/percent of special population enrollees by college against total 
population 

2. Number/percent of special population enrollees by program against 
total population 



2 Minnesota technical colleges are still struggling with the selection and implementation of 
a standard and measure that measures academic gain. This standard will be phased in over 
time. 
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3. Graduates' satisfaction of the training program(s) 

4. Number/percent of graduates available for placement that are placed 
in related employment 

5. Number/percent of special populations graduates available for 
placement that are placed in related employment 

6. Retention/nonretention rate of students in the program 

The above measures are referred to as effectiveness measures. In 
addition, the SBTC also requires efficiency measures (e.g., cost, student- 
instructor ratio) designed to complement the effectiveness measures. The 
efficiency measures expand the accountability system to provide each 
technical college with a framework for a complete and objective progress 
assessment. 

The Minnesota system does not have an established core of performance 
standards. Instead, each college must set their own standards based on 
their respective geographic draw of specific populations. For example, if 
through data analysis a college discovers it is underserving LEP students 
upon comparing enrollment data on that group to the proportion of LEP 
students in the college's service area, corrective action must be taken. Thus, 
realistic performance standards can be set by using the data collected on a 
particular group of students (e.g., 5% in the area/s served; 2% enrolled in 
the college). Technical colleges exercise autonomy in determining their 
standards. However, the standards in their local application become the 
college's standards upon local application approval. 

A team of seven professionals led by Ann Wood, SBTC's Process 
Improvement Specialist, who is charged with managing the implementation 
of the Perkins Act, oversees the accountability program. Each of the SBTC 
colleagues acts as a liaison to two or three of the 18 technical colleges in 
Minnesota by providing technical assistance on implementing the 
accountability system. Early in the program, the liaisons and technical 
colleges' personnel participated in inservice sessions to familiarize 
themselves with the Perkins Act, the Minnesota accountability system, and 
the parameters of their job. An ongoing communication between the 
technical colleges' personnel and SBTC staff continues to strengthen 
accountability efforts in the state. 
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As intended by the legislation, the Minnesota accountability system is 
the framework for each college's annual self-evaluation. By the end of the 
1993 fiscal year, each of Minnesota's technical colleges completed an 
evaluation of their technical programs (e.g., drafting, auto body mechanics, 
electronics). As a result of these evaluations, colleges not making substantial 
progress toward meeting the established standards are required to develop 
program improvement plans predicated on the data they have gathered. 
The data is also disaggregated to ascertain how special populations students 
are performing in relation to the general population. 

The practicality of the Minnesota system is also reflected in their local 
plan requirement. Colleges must use the measures data to support the local 
application for Perkins funding for the following year. The local plan must 
also contain a plan for correcting any deficiencies. (See Appendix B for 
information on the Minnesota SBTC local application requirements.) 

As instructors and administrators perform their respective 
responsibilities required by the accountability system, they have learned to 
appreciate and value what each of them can contribute in responding to the 
diverse needs of all their students. Cognizant of the support they have 
from the SBTC, they are able to improve their institutions not only for their 
students but also for themselves. Bringing about educational change through 
the implementation of the accountability system in Minnesota has truly 
been a collaborative effort. 



INCENTIVES OR ADJUSTMENTS 

Incentives or adjustments within accountability systems designed to 
encourage service to targeted groups or special populations must be 
addressed. This section discusses how incentives and adjustments can 
promote access and success for members of special populations. In addition, 
examples of incentives and adjustments are provided. 

Incentives 

Recognizing that accountability systems may induce programs to only 
recruit higher achieving students, Congress stipulated that each state's 
system of standards and measures must include incentives or adjustments 
that encourage service to members of special populations. As Hoachlander 
et al. ( 1 992) point out, however, neither the law nor the regulations specify 
what those appropriate incentives or adjustments should be. 
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Incentives can be implemented at the local or state level. For example, 
states can reward local districts or agencies who have large numbers of 
special populations that are performing at or above the standards set. 
Additionally, local recipients with greater numbers of special population 
students that are successful in meeting the standards may be rewarded with 
the following: 

• Supplementary equipment 

• Added points in evaluating grant proposals 

• Relief on some state or federal regulations 

Furthermore, local authorities may give incentives to schools or teachers 
of programs that fulfill the targeted standards. Both local- and state-level 
incentives can be monetary or nonmonetary in nature. While monetary 
incentives may be more appealing, the availability of funds may pose a 
difficulty for local and state authorities. 

Hoachlander et al. ( 1992) discuss four forms of nonmonetary incentives: 
(1) value-added measures or gains, (2) standards based on relative 
improvement, (3) regulatory relief, and (4) awards and recognition. They 
suggest that including measures of gains in accountability systems can 
become powerful incentives to serve special populations. This means a 
program could be evaluated based on the absolute gain in learning that its 
students achieve. For instance, a program serving a large number of students 
who do not possess the identified basic skills at the beginning of the program 
is more likely to achieve greater gains with these students than with students 
who are already performing at or above the standard. 

Incentives can also be stated in the form of relative improvement. 
Programs that are not making substantial progress or are operating below 
the standard that has been set need not be identified as poor performers; 
instead, states can set standards for improvement, which, if met, will be 
considered sufficient progress. 

Another way to introduce incentives in an accountability system is 
regulatory relief such as relieving districts from some regulations or reviews 
because they have met certain standards. Lastly, states can grant 
nonmonetary incentives such as awards and recognition for programs 
serving special populations and performing well above state standards, as 
well as those making above average progress. Nonmonetary incentives are 
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low cost and can help states publicize accountability and program 
improvement (Hoachlander et al., 1992). 

Adjustments 

In addition to incentives, Hoachlander et al. (1992) note that 
programmatic adjustments are one of the best approaches to promote access, 
successful program completion, and job placement for members of special 
populations. Program adjustments to accommodate special learning needs 
of students have become standard practice in most K-12 grade school 
systems and community colleges. Vocational funds and other funding 
sources have been used extensively for tutors, equipment modification, 
resource teachers for students with special needs, cl:Ud care, teacher aides, 
and a variety of other adjustments to programs. Hoachlander et al. encourage 
vocational educators to also consider using federal funding to reduce class 
size in order to better compensate for the extra instructor time required by 
students with special needs. They also suggest reducing teachers' 
responsibilities by assigning greater weights to students with special needs 
who demand extra assistance. For example, a student with disabilities could 
be counted 1.5 students. If the standard class size in one school is 20 
students, a teacher with 4 students with special needs would actually have 
a full class load of 18 students (4 x 1.5 = 6 + 14 nonspecial population 
students). Weighting outcomes can be an effective means of adjusting 
standards to reflect differences among programs with a concentration of 
students from special populations. Hoachlander et al. explain weighted 
outcomes and indices in detail. 

Team teaching is another suggested program adjustment. Inclusion 3 of 
all students into regular programs has become accepted as an effective 
educational strategy for students who have been identified for special 
education programs. Similarly, the "detracking" of students who have been 
placed in low-level classes has proven to be a very effective school reform. 
When students from special education programs enter regular classes, their 
teachers may team with the regular classroom teachers to work with all 
students. 



? There is evidence that inclusion works where there is adequate inscrvice and preparation. 
In all cases, it is imperative that teachers and staff receive intensive inscrvice on cooperative 
learning, team teaching, and applied learning prior to implementing a full inclusion policy. 
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Students with special needs may require additional time to complete 
programs and may even need a customized instructional program. When 
secondary school students are involved, those cases should be identified by 
the Individualized Education Plan (IEP) team, which will include the 
appropriate teachers, the student, the parents, and others as are required. 
The IEP should reflect the entire education plan, including the vocational 
education component, a transition plan, and a very specific career goal 
(e.g., tire technician) attainable within the competencies of a recognized 
program. 

Section 115(b) of the Perkins Act specifies that each state's 
accountability system must include incentives or adjustments that are 
designed to encourage service to members of special populations. In Illinois, 
the State Board of Education, in cooperation with the Illinois Community 
College Board, has proposed a comprehensive set of performance standards 
and measures for all groups of students, including targeted groups and special 
populations. Illinois incorporates the participation rates for targeted groups 
and special populations into its accountability system. Additionally, Illinois 
administrators realize the significance of implementing appropriate 
adjustment techniques to supplement the targeted performance standards 
and measures. The adoption of the "improvement approach" over the 
"competitive approach" is highly recommended in setting local performance 
goals, as well as in developing and implementing adjustments in Illinois. 
Under this system schools are not in competition with each other, but the 
system places each school in competition with itself by evaluating its 
performance in comparison to the previous year's performance. Using the 
state standard as the ideal level of performance for vocational programs, 
each school's performance goal for a given measure and year is based on a 
percentage reduction of the difference between its actual performance during 
the previous year and the state standard. For example, suppose the standard 
is 100% and the previous year's performance for XYZ school was 80%. If 
the state requires a 10% reduction of the difference between the standard 
and last year's performance as a strategic goal, the performance goal for 
the school would be 82% (80 + {20 x .10}) (Baj & Sheets, 1993). 

The following are some examples of program adjustments taken from 
South Carolina's System of Performance Standards and Measures: 

• Providing release time for inservice training for teachers of special 
populations students to provide them with information relative to 
prerequisites of occupational education courses. 
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• Providing release time for inservice training for teachers of occupational 
education courses to enable them to gain an understanding of how to 
teach special population students. 

• Scheduling adequate coordination time for teachers of occupational 
education, teachers of students with disabilities, and academic teachers 
who teach students with disabilities to meet on a periodic basis to plan 
the activities of the students with disabilities. 

• Incorporating a consulting teacher model with the teacher of students 
with disabilities serving as the consulting teacher to occupational 
teachers. 

• Developing a preoccupational/exploratory course at a pre-secondary 
level (where one does not currently exist) to. enable all special 
populations students to receive hands-on experience in a variety of 
occupational course offerings* 

• Arranging for nontraditional role models, both male and fern le, to 
provide job shadowing experiences and opportunities, to speak at career 
fairs or classrooms, or to serve as mentors. 

• Providing equipment modification as needed. 

• Using uniquely developed methods of adjustments that are locally 
appropriate. 

LOCAL PROGRAM IMPROVEMENT PLANS TO IMPROVE 
OUTCOMES FOR MEMBERS OF SPECIAL POPULATIONS 

Special populations must be included in the total population evaluated 
through the state's accountability system developed under Section li5 
mandates. If as a result of the evaluation, all students, including members 
of special populations, do not meet the standards (or do not show substantial 
progress toward meeting the standards), the local recipient must develop a 
program improvement plan for the succeeding school year. The plan must 
include strategies designed to achieve progress in improving the 
effectiveness of the program and, if necessary, a description of strategies 
designed to improve services provided to individuals who are men bers of 
special populations (Section 117(b)). 
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Review of Evaluation Findings 

When local recipients use their state's system of standards and measures, 
sufficient data should be collected to identify, by gender, the following 
members of special populations separately: 



Students with disabilities 
Students who are educationally disadvantaged 
Students who are economically disadvantaged 
Students of limited-English proficiency 

Students who participate in programs designed to eliminate sex bias 
Students in correctional institutions 

A review of student enrollment data for vocational and technical 
education should be made to determine the level of access of members of 
special populations enrolled in vocational education programs. Data should 
also show that enrollment patterns of special populations in vocational 
education programs are approximately equal to their enrollment in the 
school/institution. 

Local recipients should also review outcome data (e.g., number of 
students successfully completing program "X," placement rate) of all 
students including those who are identified as members of special 
populations. The review of the outcomes for each identified group of special 
populations will determine their progress and the need for initiating 
additional supplementary services and other activities necessary for 
improving those outcomes. Tor instance, a program with a number of 
Hispanic American studer.cs would want to look at comparative data between 
these students and othe r . groups. Are these students completing high quality 
programs? Are they being placed in jobs for which they trained? Do males 
and females participate in nontraditional occupational training programs? 

SECTION 118: CRITERIA FOR SERVICES 
AND ACTIVITIES FOR INDIVIDUALS 
WHO ARE MEMBERS OF SPECIAL POPULATIONS 

The Perkins Act specifies that states ensure equitable participation for 
members of special populations in vocational education programs. The 
assurances of equal access to recruitment, enrollment, and placement for 
special populations stipulated in Section 1 1 8 are criteria for states to consider 
when developing their program improvement plans. 
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Section 118, Criteria for Services and Activities for Individuals Who 

Are Members of Special Populations, of the law states 

(a) ASSURANCES OF EQUAL ACCESS FOR MEMBERS OF SPECIAL 

POPULATIONS. The State board, in its State plan, shall provide 

assurances that 

(1) individuals who are members of special populations will be 
provided with equal acce.;s to recruitment, enrollment, and 
placement activities 

(2) individuals who are members of special populations will be 
provided with equal access to the full range of vocational education 
programs available to individuals who are riot members of special 
populations, including occupationally specific courses of study, 
cooperative education, apprenticeship programs, and, to the extent 
practicable, comprehensive career guidance and counseling 
services, and shall not be discriminated against on the basis of their 
status as members of special populations 

(3) (A) vocational education programs and activities for individuals 

with handicaps will be provided in the least restrictive 
environment in accordance with section 612(5)(B) of the 
Education of the Handicapped Act and will, whenever 
appropriate, be included as a component of the individualized 
education program developed under section 614(a)(5) of such 
Act 

(B) students with handicaps who have individualized education 
programs developed under section 6 14(a)(5) of the Education 
of the Handicapped Act shall, with respect to vocational 
education programs, be afforded the rights and protections 
guaranteed such students under sections 612, 614, and 615 of 
such Act 

(C) students with handicaps who do not have individualized 
education programs developed under section 614(a)(5) of the 
Education of the Handicapped Act or who are not eligible to 
have such a program shall, with respect to vocational education 
programs, be afforded the rights and protections guaranteed 
such students under section 504 of the Rehabilitation Act of 
1973 and, for the purpose of this Act, such rights and 
protections shall include making vocational education 
programs readily accessible to eligible individuals with 
disabilities through the provision of services described in 
subsection (c)(3) 
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(D) vocational education planning for individuals with handicaps 
will be coordinated between appropriate representatives of 
vocational education, special education, and State vocational 
rehabilitation agencies and 

(E) the provision of vocational education to each student with 
handicaps will be monitored to determine if such education is 
consistent with the individualized education program 
developed for such student under section 614(a)(5) of the 
Education of the Handicapped Act, in any case in which such 
a program exists 

(4) the provision of vocational education will be monitored to ensure 
that disadvantaged students and students of limited English 
proficiency have access to such education in the most integrated 
setting possible and 

(5) (A) the requirements of this Act relating to individual who are 

members ol special populations 

(i) will be carried out under the general supervision of 
individuals in the appropriate State educational agency 
or State board who are responsible for students who are 
members of special populations and 

(ii) will meet education standards of the State educational 
agency or State board and 

(B) with respect to students with handicaps, the supervision carried 
out under subparagraph (A) shall be carried out consistent 
with and in conjunction with supervision by the State 
educational agency or State board carried out under section 
612(6) of the Education of the Handicapped Act 

(b) PROVISION OF INFORMATION. 

(1) Each local educational agency shall provide to students who are 
members of special populations and parents of such students at 
least 1 year before the students enter or are of an appropriate age 
for the grade level in which vocational education programs are 
first generally available in the State, but in no event later than the 
beginning of the ninth grade,, information concerning 

(A) the opportunities available in vocational education 

(B) the requirements for eligibility for enrollment in such 
vocational education programs 

(C) specific courses that are available 
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(D) special services that are available 

(E) employment opportunities and 

(F) placement 

(2) Each eligible institution that receives assistance under title II shall 
provide the information described in paragraph (1) to each 
individual who requests information concerning or seeks admission 
to vocational education programs offered by the institution and, 
when appropriate, assist in the preparation of applications relating 
to such admission 

(3) The information provided under this subsection shall, to the extent 
practicable, be in a language and form that the parents and students 
understand 

(c) ASSURANCES. 

Each eligible recipient that receives assistance under title II shall provide 
assurances that such eligible recipient shall 

(1) assist students who are members of special populations to enter 
vocational education programs and, with respect to students with 
handicaps, assist in fulfilling the transitional service requirements 
of section 626 of the Education of the Handicapped Act 

(2) assess the special needs of students participating in programs 
receiving assistance under title II with respect to their successful 
completion of the vocational education program in the most 
integrated setting possible 

(3) provide supplementary services to students who are members of 
special populations, including, with respect to individuals with 
handicaps 

(A) curriculum modification 

(B) equipment modification 

(C) classroom modification 

(D) supportive personnel and 

(E) instructional aids and devices 

(4) provide guidance, counseling, and career development activities 
conducted by professionally trained counselors and teachers who 
are associated with the provision of such special services and 

(5) provide counseling and instructional services designed to facilitate 
the transition from school to post-school employment and career 
opportunities 
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(d) PARTICIPATORY PLANNING. 
The State board shall 

(1) establish effective procedures, including an expedited appeals 
procedure, by which students, parents, teachers, and area residents 
concerned will be able to directly participate in State and local 
decisions that influence the character of programs under this Act 
affecting their interests and 

(2) provide technical assistance and design such procedures to ensure 
that such individuals are given access to the information needed to 
use such procedures 

Program Improvement Plan Activities 

Under Section 1 18 of the Perkins Act, local and state recipients have to 
provide assurances that members of special populations will have equal 
access to vocational programs and will be given necessary assistance to 
succeed in these programs. When a program fails to meet standards and 
has to develop a program improvement plan, it may have failed to provide 
Section 118 assurances. The assurances that follow should be reviewed 
and incorporated. 

Equal Access 

Members of special populations must be given equal access to vocational 
education programs through recruitment, enrollment, and placement 
activities. Access must be provided to all courses of study offered to students 
who are not members of special populations (Section 118(a)). A good 
guideline to follow is that vocational class enrollments will have 
approximately the same makeup as the total student body enrollment. 

Provision of Information 

Students who are members of special populations and their parents 
must be furnished information about the availability of vocational programs 
and services provided by the local educational agency. This information, 
to be provided at least one year before the students are eligible for vocational 
education, must include 

• details about the opportunities available in vocational education, 

• eligibility requirements for vocational education programs, 

• specific courses available, 

• employment opportunities, and 

• placement. (Section 1 18(d)) 
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Vocational Assessment 

An assessment of the special needs of students participating in 
vocational programs can identify individual strengths and weaknesses as 
well as education, training, and placement needs. Based on this data, the 
needs for program placement and supplementary services can be identified 
(Section 118(c)(2)). 

Supplementary Services 

Students who are members of special populations, including individuals 
with disabilities, must be provided supplementary services. Supplementary 
services that may be provided to students with identified needs include 

curriculum modifications, 
equipment modifications, 
classroom modifications, 
supportive personnel, and 

instructional aids and devices. (Section 118(c)(3)) 

Guidance, Counseling, and Career Development Activities 

Guidance and counseling activities should be provided to members of 
special populations to assist them in making and implementing informed 
educational/career choices. Members of special populations should have 
access to subject matter that provides them with career awareness, career 
planning, career decision-making, and placement skills (Section 1 18(c)(4) 
and (5)). 

Participatory Planning 

Each state should have effective procedures for developing vocational 
programs at the local and state level. Students, parents, teachers, and area 
residents should be able to participate in this program development process 
(Section 118(d)). 



SAMPLE PROGRAM IMPROVEMENT ACTIVITIES 

Program improvement activities for members of special populations 
enrolled in vocational education courses should be jointly planned with 
other school personnel who support these students in academic courses. 
For example, personnel who work with special education, Chapter I, English 
as a Second Language, and any other support persons should be included 
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in planning supplementary services for members of special populations to 
eliminate duplicated services. The following are some activities that should 
be considered when planning program improvement for members of special 
populations: 

• Provide vocational and academic teachers who teach students enrolled 
in vocational education with quality preservice and inservice training 
in working with all members of special populations. 

• Review enrollment figures and implement policies and procedures, as 
needed, to assure members of special populations equal access to all 
vocational education programs, including programs leading to high skill/ 
high wage occupations. 

• Analyze the local data showing participation in vocational education 
and outcomes of members of special populations. If the outcomes for 
members of special populations do not meet expectations, determine if 
sufficient supplementary services have been provided. When services 
are insufficient or not provided, additional services should be initiated. 

Questions to be asked should include, but not be limited to, the 
following: 

• Have sufficient assessments (vocational assessment) of students' 
interests, strengths, and abilities been made to determine the 
appropriate vocational program placement and need for 
supplementary services? 

When the vocational assessment process needs strengthening, local 
schools may consider implementing the three levels of assessment 
developed in Texas in 1980 and now used extensively by many 
states. 

Level I Vocational Assessment is designed to gather pre-existing 
student information. Information on attendance, grades, previous 
testing scores, previous work experience, prevocational classes, 
and special considerations for placement in educational programs 
should be available. As a rule, students with disabilities have 
extensive test data which is often sufficient without further testing. 
Student, parent, and/or teacher interviews may also be used to obtain 
information during the Level I stage. 



Conduct quality 
preservice and insennce 
training. 

Review enrollment 
figures. 



Analyze participation 
data and outcomes. 



Part 1: Sample Program Improvement Activities • Page 25 



NCRVE, MDS-748 



During Level II Vocational Assessment, additional information is 
gleaned from further assessments through paper and pencil type 
assessment instruments, performance tests, as well as measures of 
dexterity, spatial ability, eye-hand coordination, strength, and/or 
perceptual abilities. Level II Assessments are usually administered 
by teachers and/or counselors in the school. 

Level III Vocational Assessment is a comprehensive process used 
only when the school needs more placement information than 
obtained from Levels I or II. The Level III Assessment may last 
from a few hours to several weeks and will incorporate assessing 
abilities/aptitudes through structured exploratory exercises or 
simulated occupational tasks/work samples. A trained vocational 
evaluator should administer Level III Assessment (Patterson & 
Mikulin, 1992, pp. 10-18). 

Has the curriculum been modified for individual members of 
special populations when applicable? Has curriculum 
modification been made in coordination with other school 
program personnel who provide services for these students (e.g., 
special education, ESL, compensatory education personnel)? 

Vocational education personnel do not have the total responsibility 
for special populations students and must work cooperatively with 
other support groups within a school or institution to prevent 
duplication of services. 

Through coordinated efforts with special education, has the 
equipment and/or the classroom been modified for individual 
students with disabilities who have a need for this service? 

Special education personnel have the expertise and funding to 
provide services to students with disabilities in all educational 
settings, including the vocational classroom. Vocational education 
should share this responsibility, primarily by providing classroom 
and equipment modification, and so on, to enable a student with a 
disability to function successfully in the vocational education 
classroom. 



ERLC 



Page 26 • Part I: Sample Program Improvement Activities 



NCRVE, MDS-748 



• Are supportive personnel available to provide the additional 
and repetitive instruction for members of special populations 
enrolled in a vocational program? 

The provision of these personnel should be a joint effort between 
vocational education, special education, ESL, and Chapter I 
resources. These supportive personnel may be used in classrooms 
to assist members of special populations to succeed in the most 
integrated setting possible. 

• Is there a need for additional instructional aids and devices 
which will accommodate diverse learning styles and will 
enhance the outcomes for members of special populations? 

Teacher training and additional instructional aids and devices will 
increase the outcomes of all students, including members of special 
populations. 

• Evaluate the effectiveness of the counseling and career development 
activities provided to all students, including members of special 
populations. Career development information and planning should be 
in place by the 7th and 8th grade to help students plan for their high 
school vocational and academic education programs. 4 Counselors are 
the key to appropriate program placement and, therefore, may need 
additional training to enable them to do the best job possible in guiding 
students to appropriate programs and in the transition from school to 
work. 

SUMMARY 

The Perkins Act of 1990 calls for the provision of quality vocational- 
technical education programs for all students, especially members of special 
populations. Part I of this handbook focused on ( 1 ) measuring the success 
of serving special groups in vocational-technical education programs and 
(2) improving programs that do not meet the established standards. 



4 Ideally, all students should have access to a comprehensive career education program 
from kindergarten through postsecondary education and beyond. 
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Clearly, a need exists for a single, comprehensive system to gather 
accountability data under the Perkins Act. In the long run, a single system 
will provide local recipients a clearer understanding of the state and federal 
priorities. A single system should also prove much easier to administer 
than a multiple system. 

It also is important that the accountability system provides information 
on gender differences and includes data on the groups the law is most meant 
to serve, which include the following: 

• Students who have limited-English proficiency 

• Students who are economically and/or academically disadvantaged 

• Students with disabilities 

• Students in programs designed to eliminate sex bias 

• Students who are incarcerated 

Thus, a comprehensive system would allow for the disaggregation of the 
data by gender and these special populations categories. 

Furthermore, accountability experts stress the importance of using 
measures of gain in addition to measures of attainment. Similarly, the 
inclusion of incentives or adjustments is critical to implementing a successful 
comprehensive system. State accountability systems are not meant to 
"punish" programs that do not meet the set standards but should serve to 
identify and correct problems. The Perkins Act prescribes activities for 
improving the education of all students including those from special 
populations. 

Part II presents examples of standards and measures that were adapted 
from various states' systems. Other standards and measures can be 
developed based on these examples. Definitions of relevant terms taken 
from the Perkins Act Final Regulations and suggested operational definitions 
gleaned from some statewide accountability systems are provided in Part 
III. 

A firm commitment to improving the education of all students is vital 
to the continued economic and social well-being of our nation. The 
vocational education community has been given an opportunity to prepare 
the new entrants into our workforce — a group that will consist largely of 
women, minorities, people with disabilities, and those who live in poverty. 
The accountability system mandated by the Perkins Act of 1990 offers the 
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means for each state to develop a system that could be the springboard for 
vast program improvement. We must meet the challenge of developing 
comprehensive systems, use them to identify our weaknesses, and then 
implement good program improvement plans. 
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EXAMPLES OF GENERAL MEASURES AND STANDARDS 
Gains in Basic and More Advanced Academic Skills 

This section presents examples of performance measures and standards 
at the secondary and postsecondary levels that were adapted from selected 
state accountability systems. Based on these examples, other measures 
and standards can be developed. The list is not meant to be exhaustive. 

Secondary Level 

(Arizona Department of Education, 1992; Connecticut State Department 
of Education, 1992) 

Suggested Measure 

Demonstrate competency gains, including student progress, in the 
achievement of basic and more advanced academic skills. 

Suggested Standards 

Basic Academic Skills 

• By the end of the academic year 1993-1994, all (100%) of the tenth 
graders in vocational programs shall have increased their entry score 
on the Test of Adult Basic Education (TABE) to at least an 85% level 
of proficiency. 

More Advanced Academic Skills 

• All (100%) of the course/program completers who do not possess the 
identified skills at the beginning of the course/program will demonstrate 
gains in the achievement of the more advanced academic skills 
associated with the course/program. The student's course/program may 
be modified through an Individualized Education Plan (IEP). 

Suggested Methods of Collecting Data 

Basic Academic Skills 

• Determine level of achievement through pre- and post-testing of tenth 
graders on the TABE and formative testing. 
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More Advanced Academic Skills 

• Determine level of achievement through activity logs, IEP, and student 
portfolio assessment. 

Target Population 

All tenth graders/all juniors and seniors enrolled in credit-bearing 
vocational courses. 

Postsecondary Level 

(Vermont Department of Education, 1993) 

Suggested Measure 

Students enrolling in postsecondary technical programs in community 
colleges who lack the basic academic skills will acquire such skills within 
the first 24 credit hours of their enrollment, so they can successfully 
participate in those programs. 

Suggested Standard 

Eighty percent (80%) of students in postsecondary technical programs 
who have been identified as deficient in the basic academic skills necessary 
to participate in a technical program will successfully complete a course of 
remediation so as to be eligible to take entry-level courses by the end of 24 
credit hours of enrollment. 

Suggested Methods of Collecting Data 

Through students' completion of special remedial courses which use 
appropriate approaches to teach the basic academic skills required by 
technical programs and through counseling on improving study skills. 

Target Population 

All students enrolling in postsecondary technical programs in 
community colleges who are deficient in the basic academic skills. 
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Occupational Competency Attainment 
Secondary Level 

(Arizona Department of Education, 1 992) 

Suggested Measure 

Students demonstrate occupational competency attainment of the core 
competencies for a specific vocational program area. 

Suggested Standard 

Eighty percent (80%) of students who have been full-time students and 
are exiting a vocational program will have reached at least entry-level 
mastery on 90% of the program's core competencies. 

Suggested Methods of Collecting Data 

Ascertain occupational attainment through daily observation and 
analysis of students' performance and utilization of a checklist of 
competencies. 

Target Population 

All secondary vocational education students. 

Postsecondary Level 

(Arizona Department of Education , 1992) 

Suggested Measure 

Demonstrate competency attainment in a specific occupational/ 
vocational program. 

Suggested Standard 

One hundred percent (100%) of the program completers will 
demonstrate attainment of at least 80% of the occupational competencies/ 
tasks associated with the program. 

Suggested Methods of Collecting Data 

Use list of competencies in evaluating achievement of program 
completers, through activity log and student portfolio assessment. 
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Target Population 

All postsecondary occupational/vocational program students. 

Job or Work Skill Attainment 

Secondary Level 

(Vermont Department of Education, 1993) 

Suggested Measure 

Students exiting vocational programs shall have reached an entry-level 
competency on the set of employability skills and work maturity skills 
identified in the workplace skills list. 

Suggested Standard 

Eighty percent (80%) of program completers will attain entry-level 
proficiency in workplace skills. (NCRVE recommends 100% standard.) 

Suggested Methods of Collecting Data 

Assess students' skills using ten employability skills identified for 
acquiring employment, through observation of students' lab performance 
and experiences, and through interviews. 

Target Population 

All senior students enrolled in vocational programs. 

Postsecondary Level 

(Vermont Department of Education, 1993) 

Suggested Measure 

Graduates of postsecondary technical education programs will achieve 
the occupational skills necessary to work in the area for which they have 
prepared. 

Suggested Standard 

Eighty percent (80%) of students who complete a degree/certificate 
program in a postsecondary technical education program will be equipped 
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with the occupational skills to obtain the type of employment they sought 
when they entered the program. 

Suggested Methods of Collecting Data 

Interview students regarding their perceptions about the program, 
through observation of daily performance of students and through use of a 
checklist of occupational skills/competencies. 

Target Population 

Students enrolled in postsecondary technical education programs. 

School Retention/Completion 

Secondary Level 

(Arizona Department of Education, 1992) 

Suggested Measure 

Vocational technical education students will continue attending or will 
complete secondary school. 

Suggested Standard 

Ninety percent (90%) of the students who enroll in a course which is 
part of an approved vocational technological program will continue attending 
or complete secondary schools. 

Suggested Methods of Collecting Data 

Through counseling, tutoring, interviews, daily observation, and surveys. 

Target Population 

All secondary students enrolled in vocational technological programs. 
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Postsecondary Level 

(Vermont Department of Education, 1993) 

Suggested Measure 

Technical programs will provide the guidance, instruction, and support 
necessary for students to pursue and complete the program with the 
academic, occupational, and workplace skills required for graduation. 

Suggested Standard 

Eighty percent (80%) of vocational technological education students 
will graduate from that program. 

Suggested Methods of Collecting Data 

Through counseling, interviews, and surveys. 

Target Population 

All students enrolled in postsecondary technological education 
programs. 



Placement 



Secondary Level 

(Vermont Department of Education, 1993) 

Suggested Measure 

Technical program completers will be able to pursue employment or 
postsecondary education which was identified as appropriate at the time of 
exit from the program. 

Suggested Standard 

At least eighty percent (80%) of the technical program completers will 
be placed in additional training or education in accredited private or public 
postsecondary institutions, military service, or employment. 

Suggested Methods of Collecting Data 

Conduct exit interview and through career planning and counseling. 
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Target Population 

All seniors enrolled in vocational technical programs. 

Postsecondary Level 

(Vermont Department of Education, 1993) 

Suggested Measure 

Students graduating from technical programs will have mastered the 
skills required to work in their field or pursue further training in their field. 

Suggested Standard 

Eighty percent (80%) of students graduating from the technical program 
will be placed in related work or further education within one year after 
their date of graduation. 

Suggested Methods of Collecting Data 

Conduct exit interview and survey and through career counseling and 
planning. 

Target Population 

All postsecondary completers of vocational technological programs. 

EXAMPLES OF MEASURES AND STANDARDS 
FOR SPECIAL POPULATIONS 

Accountability systems should underscore high expectations for all 
students. As a general rule, the same core of performance measures and 
standards should apply to all students, including those who are members of 
special populations. It is important, however, to determine how special 
populations are performing in relation to the established general measures 
and standards. In order for local recipients to comply with Section 1 18 of 
the Perkins Act, disaggregated information will be needed to ascertain 
appropriate approaches that will encourage and increase participation, 
completion, and successful job placement of students from special 
populations. The following measures and standards for students who are 
members of special populations are intended to be examples to assist 
educators in addressing Section 118 requirements and eventually, make 
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necessary programmatic adjustments that will promote access and successful 
program completion for special populations. 

Equal Access 5 
(Hoachlander, Levesque, & Rahn, 1992) 

Suggested Measure 

Special populations students will be recruited in different vocational- 
technical programs. Data collected for all measures and standards and 
program enrollment shall have been disaggregated by special populations 
categories. 

Suggested Standard 

The proportion of special populations learners enrolled in vocational- 
technical education programs will steadily increase. The targeted learner 
participation rate for vocational-technical programs should at least be 5% 
above its 1991-1992 baseline rate or in proportion to their representation 
in the school population, whichever is less. The performance of special 
populations will be disaggregated for each measure and standard based 
on sex, ethnicity, and special populations categories. 

Enrollment 6 

(Hoachlander etal., 1992) 

Suggested Measure 

Special populations students will be actively recruited and enrolled in 
vocational-technical education programs. 

Suggested Standard 

The percentage of special populations students enrolled in vocational- 
technical programs will be equal to or greater than the percentage of special 
populations students in the total ninth through twelfth grade high school 
population. 

5 The measures and standards for equal access were adapted from Colorado's System of 
Performance Measures and Standards while the measures and standards for enrollment were 
adapted from Michigan's System of Performance Measures and Standards. 

6 The measures and standards for equal access were adapted from Colorado's System of 
Performance Measures and Standards while the measures and standards for enrollment were 
adapted from the System of Performance Measures and Standards from Michigan. 
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Gains in Basic and More Advanced Academic Skills 

Secondary Level 

Suggested Measure 

Special populations students enrolled in vocational-technical education 
programs shall make satisfactory progress toward achieving mastery of 
those basic and more advanced academic skills that are specific to each 
program on the test (e.g., California: High School Performance Test, North 
, 'Carolina: Educational Literacy Test. Local agencies can choose or develop 
an instrument.) 

Suggested Standards 

Basic Academic Skills 

• One hundred percent ( 100%) of special populations students from eighth 
to twelfth grade shall demonstrate gain by scoring X or above on the 
test of basic academic skills. 

More Advanced Academic Skills 

• The course/program completers from among the special populations 
groups who lack the identified skills at the beginning will show gains 
in the achievement of the more advanced academic skills associated 
with the course/program by scoring X or above on the test on advanced 
academic skills. The student's course/program may be modified through 
an Individualized Education Plan (IEP). 

Postsecondary Level 

Suggested Measure 

Special populations students enrolling in postsecondary technical 
education programs at (name of community college) who lack the basic 
academic skills will successfully complete the required academic courses 
or sequence of courses. 

Suggested Standard 

Eighty percent (80%) of special populations students in postsecondary 
technical programs who were identified as deficient in the basic academic 
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skills necessary to participate in a technical program will successfully 
complete academic courses and remediation courses (if needed) so as to be 
eligible to take entry-level courses. 

Occupational Competency Attainment 

Secondary Level 

Suggested Measure 

Special populations students shall achieve X% of the core competencies 
for a specific program on the test (e.g., competency checklist that can be 
developed; Rhode Island state-approved criterion-reference competency 
exams). 

Suggested Standard 

One hundred percent (100%) of the program completers will 
demonstrate attainment of at least 80% of the occupational competencies 
associated with the program. 

Postsecondary Level 

Suggested Measure 

Special populations students enrolled in postsecondary vocational- 
technical education programs will achieve X% of the required competencies 
for that specific vocational program area on the test (e.g., SCANS- 
competencies; Oklahoma occupational test developed by Oklahoma that 
includes written and performance components). 

Suggested Standard 

X percentage of special populations will successfully attain X% of the 
core competencies for that specific program area. 



4€ 

Page 40 • Part II: Examples of Measures and Standards for Special Populations 



NCRVE, MDS-748 



Job or Work Skill Attainment 

Secondary Level 

Suggested Measure 

Program completers shall have successfully mastered all the 
occupational competencies/skills (e.g., "Work Keys" from ACT, "Workplace 
Readiness" from Agency for Instructional Technology) necessary to work 
in the occupational area for which they have prepared. 

Suggested Standard 

Eighty percent (80%) of the vocational-technical education program 
completers will be equipped with the occupational skills necessary to obtain 
the type of employment they sought when they entered the program. 

School Retention/Completion 7 

(Hoachlander et al., 1992) 

Suggested Measure 

Special populations students enrolled in vocational-technical education 
programs will complete secondary/postsecondary school. 

Suggested Standard 

The percentage of special populations students completing vocational- 
technical programs will be equal to or greater than the percentage of 
nonspecial populations students completing vocational-technical programs. 



7 The measures and standards for school retention/completion and placement were adapted 
from Michigan's System of Performance Measures and Standards. 
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Placement 8 

(Hoachlanderetal., 1993) 

Suggested Measure 

Special populations graduates of vocational-technical education 
programs will be able to pursue the employment identified as appropriate 
for them and for which they have prepared. 

Suggested Standard 

The percentage of special populations students placed will be equal to 
or greater than the percentage of nonspecial populations students who are 
placed. 



8 The measures and standards for school retention/completion and placement were adapted 
from Michigan's System of Performance Measures and Standards. 
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GLENDALE UNION HIGH SCHOOL DISTRICT: 

The District of Choice ... Not Circumstance! 

The Glendale Union High School District began with one school in 1911. Today, 
approximately 13,300 students from diverse cultural backgrounds (grades 9-12) attend 
nine high schools: Apollo, Cortez, Glendale, Greenway, Independence, Moon Valley, 
Sunnyslope, Thunderbird, and Washington. They enjoy a comprehensive offering of 
academic courses; varied vocational and special education programs; well-established 
athletic activities; and many cultural, club, and special interest opportunities. The district 
serves a 60-square mile area which includes parts of Phoenix and Glendale. 

The Glendale Union High School District's teachers, administrators, business 
people, retirees, and parents share one vision — To empower all students for the choices 
and challenges of the 21st century. There is one total system for continuous 
improvement of student learning. Its components tightly knit the alignment of curriculum 
and the assessment and delivery of instruction. Features of the system include teacher- 
initiated instructional planning, district developed criterion-referenced tests, data-based 
decision making, and dissemination of student achievement data to the public. Ongoing 
staff development and technical assistance are provided to staff. Glendale Union High 
School District is recognized as a model district for its student performance-based and 
results-oriented program. 
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Glendale Union High School District 

VOCATIONAL/TECHNOLOGICAL EDUCATION 
Performance Standards 



Local Evaluation Team 
Resource Guide 
Spring, 1993 
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PERFORMANCE EVALUATION OVERVIEW 



The Annual Performance Evaluation applies to any eligible recipient who receives funds 
through the Carl D. Perkins Vocational and Technical Education Act Amendments of 
1990 (P.L. 101*392). Eligible recipients must evaluate all vocational technological 
programs regardless of funding sources. A program is defined as a coherent sequence 
of instruction. Each component of the instructional sequence will be evaluated including 
integrated academic courses. 

Each district offering vocational and technological education programs must establish a 
continuous improvement process for improving performance related to the Annual 
Performance Evaluation. The continuous improvement process is to be described in the 
district local application and, as a minimum, include: 1) The people involved; 
2) Annual Performance goal setting; 3) In-course progress checks and adjustment plan; 
and 4) Improvement planning based on the Annual Evaluation results. 

Each district offering vocational technological education programs must designate a 
vocational technological education program evaluation coordinator and assure his/her 
attendance and participation it an Annual workshop conducted by the Arizona 
Department of Education to disiftminatf, the annual program evaluation procedures. 

Each district must conduct the Annual Performance Evaluation, according to the schedule 
and utilizing the core standards, measures and methodology prescribed by the state Board 
for Vocational and Technological Education. As a minimum, the methodology will 
provide that Local Educational Agencies shall: 

a. Comply with the Annual Performance Evaluation Procedures as stated in the 
Arizona State Plan for Vocational Technological Education. 

b. Review each program area with the full and informed participation of 
repres e nta tives of individuals who are members of special populations, to: 

1) Identify and adopt strategies to overcome any barriers which are 
resulting in lower rates of access to vocational technological education 
programs or success in such programs for individuals who are members 
of special populations; and 

2) Evaluate the progress of individuals who are members of special 
populations in vocational technological education programs assisted under 
the Carl D. Perkins Vocational and Technical Education Act. 
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c. Evaluate the progress of programs areas in providing vocational technological 
education students with strong experience in and understanding of all aspects of 
the industry the students are preparing to enter. 

d. Local program evaluation coordinator must select and train teams of individuals 
who will review the local program evaluation and, if necessary, develop the local 
improvement plan. Each team must include, but is not limited to, at least one 
representative from each of the following groups: 

1) Special Populations 

- Handicapped (students/parents/advocate) 

. Disadvantaged (students/parents/advocate) 

- LEP (students/parents/advocate) 

- Sex Equity (students/parents/advocate) 



2) Business/Industry (as necessary to insure representation of each of 
the programs evaluated), this may include a members) of the 
current advisory committee. 

3) Teacher (as necessary to ensure representation of each of the 
programs evaluated). 

5. Local Educational Agencies may request modifications to the prescribed Core Standards 
and Measures. Upon approval of the state Board for Vocational and Technological 
Education, modifications to the Core Standards and Measures may be made based on 
economic, geographic or demographic factors, or the characteristics of the populations 
to be served. 

6. By the date established by the Arizona Department of Education, each district must 
report the evaluation results to the Arizona Department of Education for review, 
recommendarions, and approval. Team members holding d i sse nt i ng views must be given 
the opportunity to express those views as an "addendum" to the evaluation report. 

7. Upon attainment of the Core Standards and Measures, the Arizona Department of 
Education will designate the program as an "approved program". 
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Upon failure to attain the Core Standards and Measures or upon request of the Arizona 
Department of Education (or on its own volition), a district must develop a Local 
Program Improvement Plan (for all or selected programs) utilizing the Arizona 
Department of Education prescribed format, methodology, and developmental time line. 

By the time line specified by the Arizona Department of Education, Local Educational 
Agencies must submit a copy of the Local Program Improvement Plan to the Department 
of Education for review, recommendations, and approval. The Local Program 
Improvement Plan may not extend beyond the date set for the next annual evaluation and 
must be approved by the local evaluation team members. Programs operating under the 
provisions of a Local Program Improvement Plan will be desi gn a t ed by the Arizona 
Department of Education as "provisionally approved." 

In the event a program area does not attain the Core Standards and Measures for two 
successive years the LEA must work jointly with the Arizona Department of Education, 
teachers, parents and students to develop a state and local joint plan for program 
improvement Minimally, each such plan shall contain: 

a. A description of the technical assistance and program activities the State will 
provide to enhance the performance of the program area; 

b. A reasonable timetable to improve the school performance under the plan; 

c. A description of vocational technological education strategies designed to improve 
the performance of tile program area as measured by the evaluation; and 

d. If necessary, a description of strategies designed to improve supplementary 
services provided to individuals who are members of special populations. 

The state and/or local joint plan may not extend beyond the date set for the next annual 
performance evaluation. 

Programs operating under a state and/or local joint plan will be d e si gn a t ed by the 
Arizona Department of Education as "Joint Plan Approved 1-3". 

In the event a program area does not attain the Core Standards and Measures for three 
successive years, the Local Education Agency must, in conjunction with the Arizona 
Department of Education, annually review and revise the joint state and/or local plan in 
order to improve performance and will continue to do so each consecutive year until the 
program sustains, for more than 1 year, satisfactory fulfillment of the Core Standards and 
Measures. 



APPENDIX A 



12. In the event a program area does not attain the Core Standards and Measures for four 
successive yean, the Local Education Agency must, in conjunction with the Arizona 
Department of Education, annually review and revise the joint state and/or local plan in 
order to improve performance and will continue to do so each consecutive year until the 
program sustains, for more than 1 year, satisfactory fulfillment of the Core Standards and 
Measures. 

13. If at any time, the state Board for Vocational and Technological Education makes the 
determination that a program area is not making a "Good Faith" effort to improve 
performance in accordance with the Local Program Improvement Plan or a Joint State 
and/or Local Program Improvement Plan, that program area will be designated as not 
approved and will retain that designation until the program area sustains for more than 
one year attainment of the Core Standards and Measures. 

14. In the event a program area does not attain the Core Standards and Measures for the fifth 
successive year, the Arizona Department of Education will designate that program as 
"not approved". Programs designated as "not approved" may continue working on the 
state and/or local joint plan, but will retain the designation "not approved" until the 
program area sustains for more than 1 year, attainment of the Core Standards and 
Measures. 

15. On an annual schedule, the results of the Local Program Evaluations will be analyzed by 
appropriate State Vocational Technological Education staff members. The findings 
generated by mis analysis will serve as the basis from which Program Improvement 
Strategies will be developed. Program Improvement Strategies may include targeting, 
focusing, or redirecting available resources to positively impact substandard performance. 
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ANNUAL EVALUATION PROCESS 
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Local Evaluation Team 

YOU are responsible for reviewing local programs and, 
if necessary, developing the Local Improvement Plan. 

How will you determine if there are any barriers prevent- 
ing students from accessing your programs? From making 
progress in your programs? 

What information about the vocational class do you want 
from teachers? Is this any different than what you need 
for other classes or programs? 



What information about the vocational class do you want 
from your school administrators? 




M. Johnson/Te»cher In-Service 3^3 
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What is the Local Evaluation Team? 



The philosophy behind Performance Standards is to establish goals for 
program improvement An annual local evaluation of the Performance 
Standards will document the progress of students and the benefits of their 
Vocational Technological Education. 

The evaluation focus should be on program improvement, not funding. 

The annual evaluation applies to all eligible recipients of VTE funds. Each 
district must designate a local evaluation coordinator who will select and 
train teams of individuals who will review the local program(s) and t if neces- 
sary, develop the Local Improvement Plan. Each team must include at least 
one representative from each of the following groups: 

Special Populations (as necessary to ensure equitable representation of 
the groups listed below, for each of the programs evaluated): 
Handicapped (students/parents/advocate) 
Disadvantaged (students/parents/advocate) 
LEP (students/parents/advocate) 
Sex Equity (students/parents/advocate); 

Business/Industry (as necessary to ensure related trade and industry 
representation of each of the programs evaluated, which may include 
a member(s) of the current advisory commitee); 

Teachers (as necessary to ensure quality classroom representation of 
each of the programs evaluated). 



M. Johnson/Tetcher In-Service 3/93 
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0 0 0 0 0 

Local Evaluation 
Coordinator 

You must abide by the terms of the grant and be ac- 
countable to your own constituents - teachers, students, 
staff, and parents. 

What do your teachers need from you to meet the satisfy 
the Performance Standards requirements? 

What information have you shared with your teachers 
about the new performance standards? Have you orga- 
nized the procedures? Do you have a schedule? 

What evidence can you provide to show the grant monies 
have a positive effect on student retention? Academic per- 
formance? Mastering occupational skills? 





M. Johnson/Teacher In-Service Wi 
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Instructions for the Local Evaluation Coordinator 

The Local Evaluation Coordinator will be instrumental in determining the 
success of the evaluation. The Coordinator's primary responsibilities are 
to -- 

Ensure that all VTE teachers and programs a re prepared for the 
evaluation process. 

Is there a District Assessment Plan? 

Has it been shared with those who will be involved? 

What types of in-service are needed to prepare for this process? 

What types of technical support are needed? 

Select the Local Evaluation Team fthere must be one for each pro- 
gram). 

There must be at least three members. 

There must be representation for teachers, special populations, and 
business and industry. 

Train th * Local Bvaim tWn T*P m 

Focus on the Local Evaluation Worksheet Key Questions, the per- 
formance standard table questions, and definitions to help train the 
teams. 

Develop uniform reporting procedures, templates, guidelines, etc. 

Encourage a constructive, resu lts-oriented evaluation. Minority Re- 
ports, an d Local imprpYtip^t Pto"* 

Provide c ontinuous feedback. 



M. Johnson/Teacher In-Service 3W 
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Vocational Teachers 

You must provide the information necessary to 

satisfy the requirements of the new performance stan- 
dards. 

Do you know what information you need as part of 
the Local Evaluation process? Is there some way to use 
existing information for this purpose? 

Where is the information you need? How can you get 
this information? 



What resources can help you collect, organize, and 
report this information? How can this information 
help your program or class? 

Can you describe positive outcomes for your students? 
Your program? For yourself? For your school? 





M. Johnson/Teacher In-Service 3/93 
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Vocational Program Teacher Responsibilities 

Although the Local evaluation will be completed at the end of the school year, 
there is important work that must be done throughout the year. The teacher in 
each VTE class has a responsibility to ensure that information and documenta- 
tion on every one of their students is collected and maintained throughout the 
year. This information will be presented and utilized during the evaluation 
process to document student access, success, and progress. 

This responsibility is consistent with normal teaching practices that require 
that teachers have in place a means to measure and document student 
progress. 

Measure 1. Gains in Basic and More Advanced Related Academic Skills 

Documentation for this may include a list of the basic and more ad- 
vanced academic skills associated with the course or program; copies or ex- 
amples of the assessment devices used to determine gain (e.g. f pre- and post- 
tests, performance assessments); and/or a list of students enrolled in each 
course, which identifies those with IVEPs and shows those who demonstrated 
gain. 

Measure 2. Mastery of Occupational Competencies 

Documentation for this may include a list of the validated competencies 
associated with the program and each course within the program; a definition 
of how a student successfully demonstrates mastery for each competency; 
and/or a list of students in each course, which identifies those with IVEPs and 
shows those who demonstrated gain. 

Measure 3. Retention of Students in School 

Documentation for this may include a listing of all students enrolled in 
the course, identifying those with IVEPs, and/or information on every student 
who withdraws from the course during the year, identifying those who have 
remained in school as well as those who have left. The key role of the teachers 
is to work with all the available support services to encourage students to 
remain in school, and to document any services provided to the student. 

Measure 4. Placement 

Documentation for this may include a list of all program completers from 
the previous year, which identifies students with IVEPs and/or the results of a 
follow-up on these students. 



M. Johnson/Teacher In-Service 3^3 
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Instructions for the Special Populations Team Member 

Each Local Evaluation Team must have at least one member representing 
special populations. This team member is to ensure that special populations 
students have full and informed participation in all VTE programs and courses. 
(See Criteria 1, 2. and 3 below.) 

Criteria 1. Has the program under review m ade an effort to identify and adopt 
strategies to overcome barriers wh ich are resulting in lower rates of access to 
VTE programs and /or increase the success in completing such programs for 
individuals who are members of special populations? 

How many special populations students, by category, are there? 

What are the requirements for eligibility? 

What is the specific coherent sequence of courses for this program? 

What special services or equipment are available? 

What employment opportunities are available upon completion of the 
program? (How? By whom? Before the 9th grade? May I tiee an example?) 

Does the instructor modify the program to meet individual goals/out- 
comes and modify the educational method of delivery of competencies based 
upon individual needs or goals? (How are needs assessed? Are written indi- 
vidual goals determined for students needs? Does each student have an IVEP? 
Does the instructor have input into the development of the IVEP? Does the 
teacher receive the IVEP?) 

Criteria 2. Does the program under review h ave an established process for 
evaluating the student's progress in reaching their established goals /out- 
comes? 

Are students with IVEPs monitored for progress toward their goals? By 
whom? How? 

Criteria 3. Dn*s the pro gram under review have a procedure to ensure that 
special populations students are m aking progress bv providing related experi- 
ences in all aspects nf the industry In which thev are being trained? 

Are students evaluated to ensure they understand concepts? How? 

Are students provided with job-related experiences? (Written test? 
Demonstration-performance? Remedial instruction? By whom?) 

M. Johnson/Teacher In-Swvice 3£>3 
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Instructions for the Business and Industry Team Member 

Each Local Evaluation Team must have at least one member representing 
the business/ industry the program is preparing students to enter. Team 
members will review data collected by the school to determine whether or 
not the program has met each of the program standards. 

The business/industry team member is responsible for determining if the 
vocational programs are providing students with strong experience in and 
an understanding of all aspects of the industry the students are pre- 
paring to enter. Additional concerns regarding specific Performance Stan- 
dards are listed below. 

Measure 1. Pains in Basic and More Advanced Related Academic Skills 

The business/industry representaive should be especially concerned 
that the academic skills identified by the school are, in fact, related to the 
occupations for which the program is designed. The representative may 
identify any academic skills areas where the school needs to do additional 
work. 

Measure 2. Mastery of O ccupational Competencies 

The business/industry representative should be especially concerned 
that the competencies are reasonable for this occupational program. Are 
the competencies validated? By whom have they been validated? Are the 
competencies appropriately divided among courses? Do the number of 
competencies identified set high expectations for the students? 

Measure 3. Retention of Students in School 

Team members should ask questions about any students who were 
not retained in school. Are the school and teacher making appropriate 
attempts to keep students in school? Are there any school-to-work transi- 
tion programs (such as cooperative education) to help keep the students in 
school? 

Measure 4. Placement 

Team members should be concerned that the placement is related to 
the vocational training received. Where students are placed and the types 
of placement assistance provided by the school should also be concerns. 



M. iohmon/Teicher In-Service 3/93 
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Performance Standards 
Vocational Education Teachers 

Suggested Documentation and information 

•Course Outlines and Structures 



•Special Needs Students List and 
Individual Vocational Education Plans (IVEPs) 



•Related Academic Skills Lists and Matrix 



•Copies of Assessment Tools Used to Measure Academic Skills 



►Occupational Competencies for Each Course 



•Assessment Tools for Measuring Student Competency 

Attainment 



•Examples of Instructional Materials, Textbooks and Support 

Materials 



^Record Keeping Systems and Materials 



•Other Information That Might Be Helpful in Understanding 
Individual Courses and the Vocational Program 
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Preparation and training are the keys to a successful evaluation. The Local Evaluation 
Coordinator will be an integral part of this entire process. Feel free to call on that person for 
guidance and ;Unficauon. Likewise, reviewing these materials, posing scenarios, and 
coordinating with the other team members will enhance the process 




REMEMBER, THE BASIS FOR YOUR 
ENTIRE EFFORT IS PROGRAM 
IMPROVEMENT FOR ARIZONA'S 

STUDENTS. 
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Local Evaluation Teams 
Vocational Education Teacher Participation 



Purpose : To provide each school the benefit of an objective perspective 
and evaluation of each vocational program and to provide the avenue 
for the cross fertilization of ideas regarding program improvement. 

Method; The vocational program area teacher representatives from each 
school will participate as a member of an Evaluation Team other than 
that of their own school. This will occur both during the on-site visits and 
the final report writing session in June. 



The following rotation wiii be followed for the Evaluation Teams for 
Spring, 1993. 



CORTEZ 



SUNNYSLOPE 



SUNNYSLOPE 



WASHINGTON 



WASHINGTON 



CORTEZ 
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Local Application 



& 



Criteria/Rating Materials 

For Carl D. Perkins Vocational Education Funds 

FY 95-96 



A basic part of tha local application approval procass is 
tha rsvisw of tha docuaants subaittad. For tha Mia&asota 
Board of Tacbnical Collagas, tbia will ba dona by a rawiaw 
taaa coaprisad of Systaa Offica aad Tacbnical Collaga Staff. 

In ordar for tba local application to ba approvad, it will 
be nacassary to obtain a total acora of at laast 332 points , 
and to raspond to aacb itaa/quastion includad in tha 
application. 



Minnesota Board of Technical Colleges 
300 Capitol Square Building 

550 Cedar Street 
Saint Paul, Minnesota 55101 
Telephone: 612/296-0679 
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Statement of Non-Discrimination 



The State Board of Technical Colleges complies with all 
federal and state laws, rules, regulations, and policies 
prohibiting discrimination on the basis of race, color, 
creed, religion, national origin, gender, age, marital status, 
status with regard to public assistance, sexual preference, 
disability, or any other recognized protected class. No 
individual shall be denied benefits or be subjected to 
discrimination in the educational process or employment. 



Inquiries regarding compliance with Title VI of the Civil 
Rights Act of 1964, Section 504 of the Rehabilitation Act of 
1973, Title IX of the Education Amendments of 1972, the 
Age Discrimination Act, the Carl D. Perkins Vocational 
and Applied Technology Education Act of 1990, or other 
applicable Minnesota statutes or federal regulations may be 
directed to Helen Henrie, Deputy Chancellor, Minnesota 
Board of Technical Colleges, 300 Capitol Square Building, 
550 Cedar Street, St. Paul, Minnesota, 55101, 612/297- 
4575. 



Upon request, this document will be made available in 
an alternate format such as large print or cassette tape. 
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INSTRUCTIONS FOR COMPLETION OF THE APPLICATION 



Ail eligible recipients of Carl D. Perkins Title II (Basic Grant) funds are required 
oy*the legislation to submit a local application as a condition of funding. Under 
z'r.is Title, Part C funds are allocated by formula and distributed to the eligible 
recipient following receipt and acceptance of the local application. 

7c the degree possible, the application should reflect the eligible recipient's 
planning for a two year period, i.e.,. F* 95 and 96. An annual update of the 

application will be required for FY96. 

Every attempt is being made to make the local application process as "user friendly" 
as possible, given the requirements of the legislation and state policy. The 
application however, is still quite complex; therefore, the following steps are 
recommended for developing the local application. 



3 . 

4 . 



8. 
9 , 

10. 
11 . 
12. 

13, 



Establish a team for developing the local application. • The team should be 
comprised of representatives from administration, supplemental services, 
support services, equity, the business office, and students. 

Identify a coordinator or "contact person" who will act as liaison on 
behalf of the team. This individual would be a communication link between 
the eligible recipient and the State Board staff who provide technical 
assistance to that college or school. 

Obtain "Preliminary Authorization to Plan" 4 from local board. 

Make the team a "working team" that meets quarterly, at a minimum, to 
monitor and evaluate progress as well as review financial status 

(expenditures) . 

Use the same team (with additional assistance as needed) to construct the 
self-evaluation and data matrices. 

Use the standards and measures data, in addition to any other available 
data, to determine what activities should be funded. 

Develop a data matrix that contains the necessary elements, yet displays 
the data in an easy-to-interpret manner. 

Keep continuity of the plan and team from one year to the next. 

The college's mission, strategic plan, and/or NCA plan should be reviewed 
and some connection should be made between these instruments and the local 

application. 

Use the criteria/rating materials as a guide in the local plan development. 
Obtain the necessary signature (s) for the cover sheet. 

Keep the local application simple and concise. The completed application, 
including narrative, budget, assurances, implementation plan, and matrix 
must not exceed 50 pages. 

By April 1, 1994, send the original and 8 copies of the local application 
to Ann Wood, Manager of Carl Perkins Activities, Department of System 
Improvement Services, Minnesota Board of Technical Colleges, 300 Capitol 
Square Building, 550 Cedar Street, St. Paul, Minnesota, 55101. When the 
local application is acceptad, confirmation will be made to the eligible 
recipient . 
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APPENDIX B 



IMPLEMENTATION PLAN FOR PROGRAM IMPROVEMENT 



Id«ntifi«d N»«d: umtiu •* a»««i«i f^uti** at**»«t« 



Prograa/S«rvic« to b« F\ind«dt 



Go* 1 I T» gravla* •a*>r«f>rl«t« fu*yla*ja*t«l f irrlaaa t* «ii «tu4*at« ta Miuri tfcalr 



OftJmCTIVSI 



ITtATBOIM 



usroMtiaiLirr/ 

TIKJLIHXJ 



1. To laeraaaa ntutioa of 
ap+cial population atudanta la 
•11 program* . 



(•) To •■ploy- tutors to ataf f raaourco rooms 

OS MOh OASfma. 

(b) to aaploy tutor* to larva in claaaaa 
high ooaeaatratioaa of apacial populatioa 
atudauta. *pproKi«*t*ly X) tutors for 
Mating strategy a b. 

(c) frovldo auparviaory •arvieaa for 
•upplaavaotal larvioaa Oapart»ant. 



1. To 
imcraaaa 
rataatioo of 
apacial 
populatioo 
ftudaata by 
S% la tha 
oollaga . 



$127, 27« 



Supplaataoeal 
luptrviiori/ 
lach Quarcar of 
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APPENDIX B 



■MM'KNDIX K: ASSURANCES OK COMPI.IANCK FORM 



Technical Collage niuret that individuals who are ■emben of the special populations identified 



in the Carl 0. Perkins Vocational 4 Applied Technology Education Act of 1990 will be provided: 



Objective 



Equal access to 
recruitaent, enrol laent 
sod placeneot activities 

for individuals: 

a. with handicaps, 



b. wuo are educationally 
and economically 
disadvantaged 
(including foster 
children) . 



of liaited English 
proficiency . 



d. who participate in 
programs deaigned to 
eliminate sex bias. 



in correct lone 1 
Institutions . 



Equal acceaa to the 
full range of 
vocational technical 
programa for 

individua Is : 

a. with handicaps 



who ere 

educationally and 
economical ly 
disadvantaged 
(including foater 
children) . 



of liaited English 
proficiency . 



who participate in 
prograaa designed 
to eliminate aex 
biaa 



in correctionel 
institutions . 



Activity 



Timeline 



Person 
Responsible 
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ASSURANCES OKCOMI'MANCK FORM 



Technical College assures that individuals who sre eembers of the special populations identified in the 



Carl D'. Perkins Vocational i Applied Technology Educst ion ^ct of 1990 will be provided: 



Objsctivs 



Activity 



Tiaslins 



Person 
Responsible 



Vocational technical 
education progress in 
the "least restrictive 
environment" for 
individuals; 

a. with handicaps. 



4. Rights and protections 
guaranteed under the 
Education of the 
Handicapped Act for 
individuals : 

a. with handicaps. 



5. Rights and protections 
guaranteed under 
Section 504 of the 
Rehabilitation Act of 
19/ 3 for individuals! 

a. with handicsps. 



ERJC L 



Coordination of 
vocational education 
planning with 
representatives of 
vocational education, 
special education and 
vocational 

rehabilitation agencies 
for individuals! 

s. with handicaps. 

Monitoring of 
vocational education 
provided to handicapped 
students to determine 
consistency with an 
individual iied 
education program when 
appropriate for 
individuals i 

a. with handicaps. 



Hon*toring to ensure 
disadvantaged/ limited 
English proficiency 
students have access to 
education in the sott 
integrated settings 
possible for 
individuals t 

b. who are 

educationally snd 
economically 
disadvantaged 
(including foster 
children) . 



of limited English 
proficiency . 
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ASSURANCKS OK COMPLIANCE FORM 



Technical College assures that individuals who sre aeabers of the special populstions identified in the 

~,rl D. Parkins Vocst tonal 4 Applied Technology Educstion Act of 1990 will be provided; 



Objective 



Activity 



Timeline 



Person 
Responsible 



Assistance in 

preparation of 
spplicstions for 
individuals : 



with handicsps. 



who are 

educationa 1 ly and 
econoaically 
disadvantaged 
(including foater 
children) . 



of Halted English 
proficiency . 



d. who participate in 
progress designed 
to eliminate *•* 
biaa . 



e. in correctional 
institutions . 

10. Information in 
underatandable 
lsnguage and form for 
individuals i 

s. with handicaps. 



who are 

educations lly and 
econoaically 
disadvantaged 
(including foster 
children) . 



of Halted English 
prof iciency . 



d. who participate in 
programs designed 
to eliminate sex 
bias. 



in correctional 
institutions . 
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ASSURANCES OK COMPUANCh FORM 



Technical College atturti that individual* who are ««mb«r* of the special population* identified in the 

Carl D. Parkins Vocational t Applied Technology Education Act of 1990 will be providsd: 



object iTe 



Aeeietanca to entering 
vocational education 
couraaa and program* 
for individual*: 

a. with handicap*. 



who are 

♦ducat local ly and 
economically 
disadvantaged 
(including foatar 
children) . 



of limited Engliah 
prof iciency . 



d. who participate in 
prograjia daaignad 
to eliminate aex 
biaa . 



a. in correctional 
inatitutiona • 

12. Aaaiatance in 
fulfilling 
transitional aervice 
requirementa for 
individuala : 

a. with handicapa. 



who are 

educationally and 
economically 
d i a ad v a nt aged 
(including foatar 
children) . 



c. of limited English 
proficiency • 



d. who participate in 
program* daaignad 
to eliminate aex 
biaa. 



in correctional 
inatitutiona. 



Activity 



Timeline 



Person 
Responsible 
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APPENDIX B 



ASSURANCES OF COMPLIANCE FORM 

Technical College ittunt that individuals who are eembers of the special population* identified in the 



Carl D. Perkins Vocational & Applied Technology Education Act of 1990 will be provided: 



1 Objective 


Activity 


Timeline 


Person 
Responsible 


I Dttdi for individual*} 
I a. with handicap*. 

Q b. who art 

fl «ducat looa lly and 

1 •coDomically 

| disadvantaged 

1 (including foster 
D children) . 

c. of limited English 
proficiency . 

d. who participate is 
program* designed 
to eliminate sex 
bias. 

e. in correctional 
inatitutions . 

14. Supplementary services 

for individuals: 

a. with handicaps. 

b. who are 
educationally and 

1 economically 
1 disadvantaged 
1 (including foster 
I children) . 

c. of limited English 
proficiency . 

d. who participate in 
programs designed 
to eliminate sex 

bi... 

e. in correctional 
institutions . 
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ASSURANCES OK COMPLIANCE EORM 



Technical College inurti that individuals who irt itaberi of the special populations identified id the 



Carl D- Perkins Vocational & Applied Technology Education Act of 1990 'will be provided; 



Objective 



Activity 



Timeline 



Person 
Responsible 



Guidance, counseling, 

and career development 

activities conducted 

by professionally 

t moid counse lors and 

teachers for 

indi vidua 1 s : 

a. with handicaps. 



who are 

educa t iona 1 ly and 
•conosici 1 ly 
disadvantaged 
(including foster 
children) . 



of limited English 
prof iciency . 



who participate in 
programs designed 
to eliminate sex 
bias . 



e. in correctional 
institutions « 

16. Counseling and 

instructional services 
to facilitate 
transition from school 
to poet-school 
employment and career 
opportunities for 
individuals : 

a. with handicaps. 



b. who are 

educationally and 
economical ly 
di s advent aged 
(including foster 
children) . 



c. of limited English 
prof iciency * 



d . who participate in 
programs designed 
to eliminate aes 

bias . 



in correctional 
institutions . 
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ASSURANCES OF COMPLIANCE FORM 



Technical College assures that individuals who are Msbtri of tha special populations identified tn the 
Carl P. Perkins Vocations! fc Applied Technology Education Act of JL 990 will be provided: 



Objective 



Activity 



TiMline | Person 

I Respons lbl« 



17. Vocational education 
prograae in the eoit 
integrated setting 
possible for 
indiv iduals : 

b. who ere 

educationally and 
economically 
d isedventaged 
(including foeter 
children) . 



c. of limited English 
proficiency . 
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